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ABSTRACT 

This paper presents a general model for conceptualizing 
and testing the interactions of individual differences and 
subject-matter characteristics with instructional methods. 
The model postulates certain ways of classifying the 
variables of interest in such investigations and of con- 
ceptualizing the cause-and-effect relationships among those 
classes of variables. Some important implications for both 
research and theory-construction are discussed, and a 
resolution of the controversy over the optimal breadth of 
instructional theories is proposed. Then the paper de- 
scribes some research methodology which includes aspects of 
research design and statistical analysis that facilitate the 
investigation of those interactions. Of particular interest 
is a variation of analysis of covariance that allows a 
continuous variable (the covariate) to be analyzed as a 
factor, complete with its interactions with all the terms " 
in the basic statistical model. 
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A Methodology For Investigating the Interactions of 
Individual Differences and Subject 
Matter Characteristics With 
Instructional Methods 

One of the most important tasks facing educators is 
the development and use of highly effective, efficient, 
and appealing methods of instruction, Hjwever, this task 
is complicated by the fact that the effectiveness of many 
methods varies depending upon such factors as the charac- 
teristics of the students or the nature of the subject 
matter content. Therefore, it is important that research- 
ers and theorists consider the conditions under which 
different methods should and should not be used. To 
facilitate the consideration o'^f such contextual conditions, 
this paper (1) presents a general model as a framework for 
conceptualizing and testing the interaction effects of 
student and subject-matter characteristics with instruc- 
tional methods, and (2) desc ihes a research methodology 
which includes aspects of research design and statistical 
analysis that help an investigator to study those 
interactions . 

The purpose of investigating the interactions of 
individual differences and subject-matter characteristics 
with instructional methods is to identify the conditions 
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under which different methods of instruction are highly 
effective. But the way we categorize those conditions and 
methods can have a large impact on the stability and useful- 
ness of the relationships that are identified between those 
conditions and methods. Therefore, a matter of great 
importance is the manner in which we define and classify 
all the methods and conditions that we wish to investigate; 
and the ultimate value of any classification scheme that we 
adopt is determined by the stability and magnitude of the 
cause-and-ef feet relationships that are found to exist among 
those categories. 

There are two factors that can influence the value of 
a classification scheme for instructional conditions and 
methods: the preciseness of definition of the categories 
and the nature of the categories. The nature of the cate - 
gQ^jgs is determined by the way in which objects, symbols, 
and events are classified. For instance, trees may be 
classified according to their age (e.g., seedling, saplihg), 
their kind of leaf (e.g., pine, deciduous), or their gen,^ 
(e.g., oak, maple). The instructional .vprld can also be' 
"sliced" in different ways. The work of M. David Meriill is 
based upon the assumption that such categories as physics, 
English, and algebra are not as useful for predicting the 
outcomes of instructional methods as are his task/content 
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classification of subject matter (e.g., remember a 
procedure,^ use a principle). (See Me. rill, Richards, 
Schmidt § Wood, 1977; Merrill § Wood, 1975.) 

With Respect to the preciseness of definition , many 
categories of methods that are frequently used in research 
and theory construction are not very useful because the 
stability of their cause-and-ef feet relationships is jeop- 
ardized by their looseness of definition. For instance, 
"lecture" vs. "discussion group", "inductive" vs. 
"deductive", and "discovery" vs. "reception" may often vary 
more within each category than between categories. In 
such cases, it is pcicessary to break down these "methods" 
into their building blocks, and to base one's research and 
theories on those more clearly-defined strategy components . 
Such an approach is the major emphasis of M. David Merrill's 
laboratory at Brigham Young University, and the results of 
his research have consequently been relatively consistent 
and "clear"--having fewer interactions and revealing some 
broad and comprehensive principles of instruction (see 
Merrill, Richards, Schmidt ^ Wood, 1977; Merrill 5 Wood, 
1975; Reigeluth 5 Merrill, Note 1). 
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A GENERAL MODEL 

Reigeluth and Merrill ( Note l) have presented a 
conprehensive model which clar>sifies ins tructiorial vari- 
ables and indicates ways i i which those classes of 
variables influence each other. First, all varMbles 
relating to instruction are categorized as-^'ither: me t>^ods 
relating to instruction, outcomes of instruction by which 
methods can be evaluated, or' conditions which influence 
the outcomes of those methods. Then, within this' framework, 
those categories are further divided as follows (<;ee 
Figure 1): (1) methods of instruction are divided into 
strategies for organizing the instruction, strategies for 
delivering the instruction to the students, and strategies 
for managing the interaction of the students with the 
instruction; and (2) conditions influencing the outcomes of 
instructional methods are classified as to which conditions 
are likely to have the greatest influence on each class of 
methods* This does not mean that those classes of condi- 
tions are the only conditions which influence each class 
of methods. The reasons for not including student charac- 
teristics in the conditions which interact most strongly 
with organizational and delivery strategies will be 
discussed later. 

Insert Figure 1 about here 
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Figure 1. The framework of a comprehensive model 
showing cla.sses of instructional variables and the major 
(but by no means only) ways those classes influence each 
other. 
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Strategies for organizing the instruction are 
sometimes referred to as "instructional strategies", and 
they have been classified as to two types (Reigeluth, Eun- 
derson, 5 Merrill, in press): (1) presentation strategies , 
which are methods for organizing the instruction on a 
single "construct" (concept, principle, etc.), and 
(2) structural strategics , which are methods for sequencing, 
^yi. hesizing, and summarizing some related constructs. 
Much more work has been done on the development of struc- 
tural strategies, although at least one major effort is 
currently under way to develop structural strategies (see 
Reigeluth, Me.rill, et • al,, Note 2 ) . Presentation stra- 
tegies include such clearly-defined, elemental methods as: 
the presentation of a matched nonexample with each example 
for concept-classification tasks, the presentation of 
divergent examples foi all tasks at the "use" level (e.g., 
concept-classification, principle-using, procedure -us ing) , 
and the presentation of mnemonics for all tasks at the 
"remember" level . 

Strategies for delivering the instruction to the 
students are usually classified as "media", but they of 
course include such media as teachers, blackboards, 
workbooks, and textbooks, in addition to such mecnanical 
media as videotapes, slides, audio tapes, and computers. 
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Strategies for managing the interaction of the 
stuaents with the instruction include such things as 
scheduling the frequency and duration of such interaction, 
increasing the student's motivation (such as contingency 
management), maintaining discipline, and even deciding 
what methods and media a student should have at a given 
time fsee below under The Breadth of Instructional 
Theories ) . 

For purposes of investigating the interactions of 
individual differences and subject-matter characteristics 
with instructional methods, a modification -if this compre- 
hensive model' (from Reigeluth, Bunderson § Merrill, Notes) , 
is more useful (see Fioure 2). Mevertheless , the distinc- 
tion among the three kinds of methods is a verv valuable 
one and will play an important part in^our later discussion. 

Insert Figure 2 about here 

The subject-matter characteristics and student 
characteristics which interest us in this model are only 
those which interact with instructional methods. For 
instance, students with higher motivation may be found to 
score consistently higher on a variety of alternative 
methods than students of low ability. That main effect is 
not of interest to us because it has no prescriptive 
power for improving instruction. On the other hand, if 
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Figure 2. A modification of the comp,rehensive model, 
adapted for its particular relevance for investigating the 
interactions of individual differences and subject-matter 
characteristics with instruct ;ial methods.' 
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high ability students score consistently higher on method A 
than on method B, while low ability students score consist- 
ently higher on method B than on method A, this interaction 
effect is of great interest because it does have prescrip- 
.tive power for the use of good methods of instruction. 
Classifications of subject matter which have similar inter- 
action effects also greatly interest us. Any classification 
of student characteristics or subj ect-matter characteristics 
is valuable to us only to the extent that it is useful for 
predicting the outcomes of an instructional strategy. 

The "outcomes of instruction" which interest us in 
this model are those which are useful for evaluating the 
relative merit of each instructional strategy. Such 
outcomes include the e tf ect iveness of the instruction (e.g., 
level of conceptual understanding, performance efficiency, 
transfer, and long-term retention), the efficienc y of the 
instruction (e.g., level of effectiveness in relation to., 
the student time and level of effectiveness in relation to 
the cost of the instruction), and the appeal of the 
instruction (how much the student likes it, independently 
of the content being taught) . 

a 

From the point of view of the researcher or of the 
"descriptive theorist", these four classes of instructional 
variables are related to each other as follows (see 

13 
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Figure 2). The student variables, the subject matter 
variables, and the method variables are all independent 
variables; and their parameters may interact to produce 
fairly consistent (predictable) effects on the outcome 
variables, which are dependent variables. 

However, from the point of view of the instructional 
designer or of the "prescriptive theorist", the four 
classes of variables are related as follows. The desired 
outcomes, the student variables, and the subject matter 
variables are all independent variables which also interact; 
but their parameters are used (thanks to the efforts of 
researchers and descriptive theorists) to prescribe good 
methods of instruction, which are the dependent variables. 
Implications of the General Model 

This general model--for conceptualizing the effects 
of student and subject-matter characteristics on instruc- 
tional methods--has important implications for both 
research and theory construction on individual differences 
and subject-matter characteristics. First, researchers 
must bq. aware of, must control, and must systematically 
describe the important variables (and their parameters) 
that were not manipulated in all of the first three classes; 
and theorists must be careful to specify the parameters • 
of all i"iportant variables in the first two classes 

14 
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(student characteristics and subject-matter characteristics) 
for which their strategies are recommended, or they should 
specify the likely outcome differences of each strategy 
for different sets of parameters of those variables. This 
implication may seem obvious, but unfortunately researchers 
and theorists alike usually fail to identify such variables 
as clearly and systematically as they should. This failure 
greatly impedes the much-needed comprehensive review and 
comparison of related research- -what Glass (1976) referred 
to as the "meta-analysis" of research. 
The Breadth of Instructional Theories 

A second implication of this general model is related 
to the recent controversy over the optimal breadth of 
instructional theories. Richard Snow, who is well known 
for his work on individual differences, stated (Snow, 1977) 

c 

that ATI (aptitude- treatment interaction) "makes general 
theory impossible"- -that instructional theories must be 
"narrow and local" to be of value. On the other hand, 
Scandura (1977) staved that instruct ional theories must 
be "brdad and compreiaens ive" to be useful. We believe both 
points of view have merit^and can be reconciled with the 
help of the comprehensive model. 

The major source of the difference in opinion over the 
optimal breadth of instructional theory can be traced to a 
broader definition by Snow ;.han by Scandura. In reference 

15 
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to our first, comprehensive model (Figure 1), Snow includes 
all three classes of methods- -organizational , delivery, and 
management- -within the scope of instructional theory; where- 
as I believe Scandura includes only organizational 
strategies (which are often referred to as instructional 
strategies--hence their exclusive inclusion in his concep- 
tion of instructional theory). This difference in the 
breadth of definition of instructional theory has such a 
strong impact on beliefs as to the optimal breadth of 
instructional theory for the following reason: research 
literature indicates very little ATI with organizational 
strategies . whereas it indicates very strong ATI with 
management strategies (hence the configuration of student 
characteristics in .the conditions section of Figure 1). 

Our experience and research results indicate that 
clearly-defined, organizational strategy components , such 
as those described by Merrill (Merrill, Richards, Schmidt § 
Wood, 1977; Merrill § Tennyson, 1977; Merrill § Wood, 1975), 
are usually best for all students or they are best for 
none.-^ For instance, wc believe that all instruction at 
the "use" level should include the following organizational 
strategy com -nents; a generality a large, number of 
examples and practice items, atlributt isolation on the 
examples and on the practice-feedback, a full range of 
divergence on the examples and practice, generality and 

16 
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instance "helps", and th? clear separation and labeling of 
each of these strategy components. Certainly, as Scandura 
maintained, "broad and comprehensive" instructional theory 
can and should be developed- - in the area of organizational 
strategies — and such theories can have high generalizability 
across schools, as well as across time. 

But Snow has an ir.portant point with respect to 
nafiagement strategies . If instruction should always con- 
tai» the same organizational strategy components, the 
important questions become: when should the student see 
examples, as opposed to the generality or some practice? 
how many practice items and examples should a student be 
^ given? and when, should "helps" or remedial loops be made 

available? Our experience and research results indicate 
that the answers to questions like these are extremely 
sensitive to individual, differences, and that they also 
vary tremendously over time as a student's understanding 
develops jRd misconceptions arise and are dispelled. But 
this problem is not so much one of a difficulty in develop- 
ing theory (i.e., identifying cause-and-ef feet relationships) 
as it is a difficulty in applying that theory. How can 
Snow or you or I possibly keep track of the momentary and 
changing "aptitudes" of each and every student and manage 
*° provide just the right kind of instruction (i.e., the 
righ'c organizational strategy component) for that moment? 

• 

o 17 
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The theory is possible ind would be generalizable across 
time and schools: that is, given certain specific combi- 
nations of conditions, a particular management strategy 
would optimize certain outcomes. But that theory could 
not be applied except perhaps on a sophisticated computer 
system. And even then, we would have serious d^^ubts about 
how good that would be for the student. Might it not be 
better for the stadent to learn for himself what organiza- 
tional strategy component s/he needs at any given time? 
Might it not be bad for the student to be exposed constantly 
to the '*low track" of the curriculum? Maybe a different 
"track'' is what s/he would need to get off the low track. 

To us. there is a simple solution to the difficulty 
of applying knowledge about management strategies and to the 
quest ionableness of whether the teacher should apply such 
knowledge (see Merrill, 1975): learner control over the 
selection of organizational strategy components. But for 
such learner control to be effective, the learner must be 
provided with a certain kind of knowledge and information 
to enable him/her to make good decisions. Should this kind 
of learner control prove to be the most effective manage- 
ment strategy with respect to the selection of organizational 
strategies, instructional theory will be tremendously 
simplified for this difficult class of instructional methods 
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(i.e., management strategies). And there are strong 
indications that this is so. The TICCIT (Time-shared 
Interactive Computer Controlled Instructional Television) 
system, developed jointly by the Mitre Corporation and 
Brigham Young University under, a grant from the National 
Science Foundation, has a special "learner control" key- 
board, such that with the press of a single button a learner 
can call up the organizational strategy component that s/he 
feels s/he would benefit most from at that moment: an 
easier practice item, the generality, a "help", a hard 
example, an easy example, etc. And the learner can maye on ' 
to the next construct (e.g., concept, principle, or proce- 
dure) whenever s/he feels ready. There is also an "advisor" 
to give the learner advice about whether or not s/he under- 
stands the construct well enough yet. The National Science 
Foundation is currently funding a project to investigate the 
effectiveness, efficiency, and appeal of such leai ;er 
control, and the results to date are very favorable. 

In summary, although it is important that the, 
prescriptions of instructional theories be "narrow and 
local" (i.e., valid for specific conditions), it is not 
necessary that tlie theories themselves.be "narrow and 
local". Organizational theories can be and are broadly 
generalizable, and management theories can be, too. The 
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major problem is that management theories cannot be applied 
effectively with respect to the selection of organizational 
strategies--and they probably even should not be so ap- 
plied- -except by the learner. 

We believe that the general model of instructional 
variables presented above helps provide a sounder basis 
upon which to conceptualize interaction experiments and to 
control confounding variables. The remainder of this paper 
presents several research methodologies that we believe will 
help to design su:h experiments and to analyz:e their data. 

SOME RESEARCH METHODOLOGY^ 

We advocate two kinds of research: (1) controlled 
experiments entai'ling the manipulation of small numbers of 
variables in fairly lean instruction, and (2) classroom • 
experiments entailing the testing of whole models of 
instruction under realistic conditions. Both kinds of 
research are necessary for the following reast)ns. 

Controlled experiments are conducted under artificial, 
laboratory-type conditions- -conditions that are carefully 
controlled so as to reduce confounding variables and to 
isolate pure effects. This kind of research is essential 
for increasing our understanding of individual strategy 
variables and how they interact with instructional condi- 
tions. However, variables may have significant effects in 

20 
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a certain direction under laboratory conditions, yet their 
significance may be greatly reduced--and the direction of 
their effects may even be reversed--under realistic instruc- 
tional conditions, due to the main and interaction effect, 
of all the other (sometimes unnoticed) variables. 

For instance, Reigeluth, Bunderson, and Merrill (in 
press) hypothesize that an S- shaped curve represents the 
relationship between the quality of instruction and the 
effectiveness of that instruction (see Figure 3). Experi- 
menters deliberately design their experimental treatments 
such that the variable(s) under investigation will increase 
the quality of the instruction from a-to-b in Figure 3 
rather than from c-to-d, so that the contribution of that 
variable to the effectiveness of instruction will be on the 
order of magnitude of w-to-x rather than y-to-z.^ In multiple, 
regression (statistics) an independent variable has a much 
higher correlation with the dependent variable if it is 
taken alone than if it is adjusted for all other indepen- 
-^pnt variables. The same is probably true of instructional 
variables such that the significance of any instructional 
variable is likely to be of a much lower order of magnitude 
in real instructional settings than in controlled 
experiments . 

Insert Figure 3 about here 
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These limitations are serious and require the 
performance of classroom experiments with realistically rich 
instructional methods under realistic instructional condi- 
tions. Such experiments must test the relative effects of 
large numbers of instructional strategy components when 
such components are all present in the instruction, because 
it is highly likely that many important ones overlap in 
function and/or interact with each other, tt is important 
not only to determine which strategy components contribute 
the most to the quality\ of the instruction, but also to 
compare their relative contributions with their costs in 
extra student learning time. 

The following should be considered in conducting 
experiments for both kinds of research. First, all of the 
variables which might possibly influence instructional out- 
comes—from Classes 1, 2, and 3 in Figure 2--should be 
carefully controlled if they are not manipulated. Special 
care should be taken to avoid .confounding among classes 
of method variables as well as within classes (e.g., to 
assure that an organizational strategy is not confounded 
with a management strategy, as well as to assure that two 
organizational strategies are not confounded). Second, a 
variety of instructional outcomes should be measured in 
any experiment. The effectiveness, efficiency; and appeal 
of the instruction are all important. And it is usually 
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valuable to investigate a variety of important measures of 
eflectiveness- -such as conceptual understanding, perform- 
ance efficiency, transfer, and long-term retention- -and of 
efficiency- -such as learning time and the monetary cost of 
the instruction. 

Methodology for Controlled Experiments 

For controlled experiments, we have used the following 
methodology with some success (see Reigeluth, 1977, for an 
example) . 

1. Plan the basic experiment . That is, decide which 
instructional strategy variables you will investigate, and 
what kind of a "basic" statistical model you will use to 
analyze the results. (All other strategy variables should 
be held constant across all treatment groups.) 

For example, Reigeluth (1977) investigated the effects 
of generalities, examples, and practice. The levels were 
absence and presence of each, giving a 2 x 2 x 2 basic 
model. 

2 ■ Decide which subject-matter characteristics (if 
any) to m anipulate . The different levels of the subject- 
matter variable can be analyzed as replications of the 
basic model by having a separate task and a separate set 
of dependent variables for each level. The students in 
each treatment group of the basic design receive all rep- 
lications (tasks) ia random order, and all of the 
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replications must be unrelated tasks. A split-plot 
statistical design can sometimes be used to make direct 
comparisons among levels of the subject-matter variable. 
But often, due to incompatabil ity of such factors as level 
of chance scores on the posttesl, no direct statistical 
comparison can be made. Nevertheless, an inspection of 
differences among replications can be very useful. 

For example, in the above-mentioned experiment, I 
used a concept -class if icat ion task, a principle-using task, 
and a procedure-using task as the three replications and 
found some important differences in the effects of the 
strategy variables for each type of s ibject-matter content. 

3. Decide which individual differences to investigate ^ 
Through stratified random sampling and/or a sufficiently 
large N, try to assure a representative range of individual 

c 

differences (within each treatment group) on e xh student 
characteristic to be investigated. (Such a balanced repre- 
sentation of student characteristics is really even more 
important when such characteristics are not investigated, 
because of probable confounding of results.) Then obtain a 
valid measure of each student characteristic of interest. 

4. Select a method of statistical analysis . We 
recommend a method of statistical analysis that is easier ^ 
to use than regression and has several other advantages. 

It is a variety of ANCOVA (analysis of covariance) in which 
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you treat the ctvariate as a factor, complete with its 
interactions with all the terms in the basic model (usually 
method variables). First, you should do a separate analysis 
for each student variable. Enter the variable as a covariate 
in each replication of the basic model; but also multiply 
that covariate by each term in the model, and enter the 
product for each multiplication as an interaction term at 
the end of the model (just before the error term) . 

• In this manner, an F-test on the covariate (the student 
characteristic) tells whether that student characteristic 
has a main effect on each dependent variable (e.g., higher 
motivation causes higher posttest scores); and an F-test 
on each interaction tells whether different levels of that 
scudent characteristic call f r different levels of your 
strategy variable(s) (e.g., *high-mot ivation students do 
better on posttest scores when they have control over their 
learning, whereas lovv-mot ivation students do better when 
there is external control over their learning). 

Second, if you suspect that two student charact^eristics 
may interact (e.g., high-motivation students may require a 
different strategy when they have low ability than when 
they have high ability), both may be entered as covariates 
in the same model, but you will in effect be doubling the 
size of the statistical model each time you add a covariate 
and all its interaction terirj. 
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In relation to ANOVA, this variety of ANCOVA allows 
you to analyze continuous variables without having to cate- 
gorize them, which would sacrifice data and power (see 
Cronbach § Snow, 19^7, pp. 60-61). And in relation to 
regression, this variety of ANCOVA makes it considerably 
easier to analyze discrete variables (usually the strategy 
variables) along with the continuous variables (usually 
the student characteris rics) and to analyze the higher-order 
interactions between the two type of variables. 
Methodology for Classroom Experiments 

For classroom experiments with realistically rich 
instructional methods under realistic instructional condi- 
tions, we recommend the performance of two types of 
experiments: (1) a c orrelational study --to determine the 
validity and importance (i.e., contribution to student 
performance) of each of a large number of instructional 
strategy components (when in combination with all the other 
components) --and (2) an experimental stujiy --to test the 
relative effectiveness and the relative cost effectiveness 
(in terms of student learning time) of the most important 
strategy components (as identified in the correlational 
study) in combination with each other, under more carefully 
controlled conditions. An alternative to performing these 
two experiments in tandem would be to use a matrix design. 
This would test indivJ .ual interactions among methods more 
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effectively; but it woul'-: requii^ an experiment of too large 
a scope and expense to be feasible for many researchers; and 
the problem of testing interactions can be largely overcome 
(see below under procedures for the experimental study). 

We recommend the following procedure for the correla- 
tional study. 

1- Plan the basic experiment . Decide which ' 
instructional strategy variables you would like to study 
together. You should -elect a relatively large number of 
variables, especially ones tha* you think are likely: to 
influence each other. In this first experiment no attempt 
need be made to investigate the interaction? of student 
characteristics (although it could be done by performing a 
complicated split-plot ANCOVA analysis of individual data 
within each segment and for each strategy-- in such a case, 
valid measures of student aptitudes at the time of the 
instruction would have to be available). 

2. Select existing instruction . Find some "segments"^ 
of classroom instruction that have already been taught and 
for which test records and the written- instructional mate- 
rials used are, still available. Select from those segments 
on the following criteria: (1) a high burden of the 
.instruction was assumed by a single source of written 
instruction, (2) there is a high likelihood that all or ' 
most of the students had already mastered all of the 
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learning prerequisites, and (3) there is a low probaoility 
of student pref amil iarity with the content of the segment. 
The segments selected should b"^ as homogeneous as possible 
on these three criteria, and there should be a fairly large 
number of segments, since segments are the observational 
units. ~ 

3- Identify performance measures . For each segment 
identify those test items which measure student performance 
on that segment. Then calcu^ e the mean score (and, if 
you wish, the standard de ation) for each segment, 

4. Rate each segment . Analyze the written instruc- 
tional materials that were used for each segment to 
determine which of the strategy components were used and 
to what extent each of those components was used. Then 
rate each segment (on perhaps a five-point scale) as to 

its level of effectiveness on each strategy component. That 
level of effectiveness will probably depend upon the "task 
level" (see Merrill, Richards, Schmidt, § Wood, 19^7) of 
the test items used in #3 above (e.g,, remember an instance, 
use a generality) . 

5. Perform the correlation . For each strategy 
component, test the rating on each segment for correlation 
with the mean test score on the same segment. The resulting 
correlation coefficient for each strategy component is a 
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ba^is (a) for inferring v/hich components significantly raise 
test scores and (b) for rank-ordering thosr. components on 
the basis of how much each contributes to student perform- 
ance, -given the simultaneous implementation of all the 
other strategy components. 

If you wish to study the i nteractions. of subject- 
matter characteristics, ycu may classify the segments 
according to type of subject-matter content and do a sep- 
arate correlation for each group of segments; or to test 
subject matter main effects and interaction effects, you 
could use the kind of ANCOVA described above, with type of 
subject-matter content as a factor and with the segment 
ratings on„ aach strategy variable as the covariate. A 
composite rating of all method variables on each segment 
could also be tested for correlation with mean test scores 
on each segment to see how much of the total variance is 
accounted for by the method variables used. 

6- Repeat for new methods, if desired . If none of 
the methods was very highly correlated, you may want to try 
to identify other strategy components, and repeat the above 
five steps with the same student performance data but with 
the new ratings for the new strategy components. 

We recommend the following procedure for the 
experimental study. 
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1) Select the independent variables . Select the 
strategy variables that were identified in the correlation 
study as contributing the most to student peiformance. 
There should probably be a maximum of eight or ten dif- 
ferent strategy variables. Also select the student and 
subject-matter \r^»riables whose interactions you would like 
to investigate, a^id find suitable measures for the student 
variables • 

2) Create the treatments . Select at least one 
segment of instruction for each level of your subject-matter 
variable, and if possible select segments that are a part 
of an on-going course. Having several segments for each 
level of your subject-matter variable will increase the 
reliability and external validity of your experiment, espe- 
cially if they ^re administered at different schools. The 
segments you select should have minimal student prefamii- 
iarity and minimal learning prere-^uisi tes that cannot be 
assumed of the students. 

Design one treatment by -including in the instruction 
on the segments you selected all the strategy components 
that you wish to investigate (and any others that you think 
l^eai-world instruction ought to have). Then create 
additional treatments by successively deleting one strategy 
component (that you wish to investigate) at a time, starting 
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with the component that contributed the least to student 
performance in the correlation study. This will result in 
the crea**on of treatments ranging from one that has just 
one of the strategy components being manipulated (the 
most important one) to a treatment that has all of the 
components being manipulated. You should also create a 
control group whicji has no instruction. 

5) Choose the dependent variables . A variety^ of 
dependent variables should be used, so as to measure the ' 
effectiveness, efficiency, and appeal of the instruction. 
Scores on ^variety of tests would represent the measures 
of the change in instructional effectiveness associated ^ 
with each strategy component and its interactions; and the ^ 
learning time data would provide a measure of the change 
in instructional efficiency associated with each strategy 
component and its interactions (effectiveness divided by- 
learning time). Thus, both student learning time and 
student posttest scores should be allowed to vary (as they 
do in most real-world instructional settings), and fhey 
should both be used as dependent variables. In considera- 
tion of Cronbach and Snow's (1977, pp. 44-45) objection to 
allowing learning time to vary, we advocate the use of an 
analysis of covariance to statistically control time to 
isolate the effects of the independent variables on post- 
test scores, and vice versa. A multivariate analysis could 
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also be used to investigate the overall effects of the 
treatments on both types of dependent variables. Of course, 
for the efficiency Tr.easure (effectiveness divided by- learning 
time), the dependent variables would not be covaried on each 

her. This choice of dependent variables will allow a 
rank-ordering of strategy components on the basis of their 
co-st-benefit value for being used in instruction for dif* 
ferent kinds of subject matter. 

4) Decide on a method of statistical analysis . There 
is only one factor for instructional methods: each treat- 
ment represents a different level of that factor. The 
significance and magnitude of the differences in the means 
and standard deviations of successive treatments (for both 
posttest scores and learning time) are the statistics of 
major interest, because you want to know which strategy 
components, when added to and in combination (interaction) 
with the other important strategy components, significantly 
increase the effectiveness, efficiency, and/or appeal of 
the instruction. The least important method variables, as 
determined by the correlation study, were added last in the 
formation of the treatments in hopes that the resulting 
treatment effectiveness means, when plotted against the 
levels of instructional method, would result in a curve 
similar to that in Figure 3- This could be tested by 
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simply plotting the treatment means or by using more 
sophisticated techniques, such as polynomials or fitting 
an exponential. 

If such a curve is not approximated, then untested 
interactions could invalidate such a different rank-ordering 
of the strategy components. Therefore, it would be advis- 
able to redesign the treatments by successively adding the 
strategy components in their order of importance as 
indicated by this most recent data: and then rerun the 
experiment. But if the curve is fairly closely approxi- 
mated, a simple ANOVA with post-hoc comparisons of adjacent 
means will provide the necessary statistics for deciding 
which strategy components, when added to and interacting - 
with the other important strategy components, do and do not 
significantly increase the effectiveness, efficiency, and 
appeal of the instruction for each type of subject matter. 

Interactions with student characteristics can also be 
investigated by using the above-mentioned variety of ANCOVA, 
using the covariate (student variable) as a factpr, complete 
with its interactions with every term in the basic model. 
Summary 

We propose that the general model of instructional 
variables presented in the first half of this paper helps . 
provide a sounder basis upon which to conceptualize inter- 
action experiments and to control confounding variables. 
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We also propose that the research methodologies discussed 
in the second half of this paper help to provide sound 
ways for designing such experiments anJ for analyzing their 



data. 
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Footnotes 

^There are some exceptions, such as generalities not 
being as effective for younger children as for more mature 
learners; but exceptions aj^pear to be few. 

^I am deeply indebted ^ Melvin Carter of the 

Statistics Department at Brii,^am Young University for his 

contributions to the statistical methodologies discussed 

herein, especially the little-known variety of ANCOVA that 

uses the covariate as a factor in the analysis, 
3 

Segment as used here refei^ to the instruction on 
a single concept, principle, etc. 
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Figure Captions ^ 

Figure 1, The framework of a comprehensive model 
showing classes of instructional variables and the major 
(but by no means only) ways those classes influence each 
other. 

Figure ^, A modification of the comprehensive model, 
adapted for ii pr^ticular relevance for investigating the 
interactions of individual differences and subject-matter 
characteristics with instructional methods. 

Figure 3. The relationship between the richness of 
instruction and its effectiveness. j*- 
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